Chapter 19

Tradi tional distributed computing is dominated by a client–server model. File systems, databases, and hypertext systems are all designed with the assumption that a few powerful and reliable servers support a large and dynamic set of clients. Servers are complex and expensive, while clients are lightweight, cheap, and simple. Servers are responsible for persistence, security, and coherence, while clients are responsible for very little. Client software often acts directly on behalf of an interactive user who steers the overall interaction. Grid computing is different: a single Grid client may harness large numbers of servers over extended time periods. For example, one client may consume more than 100,000 CPU-hours in one week on systems distributed worldwide (443) (see also Chapter 10). As a consequence, both Grid clients and servers are multiplexed, multiprotocol, and multithreaded. This organization could be termed peer-to-peer (Chapter 29) to indicate that participants are equals, although it need not be the case that participating systems are poorly connected, particularly unreliable, or mutually untrusting, as is often assumed in peer-to-peer computing. The key point is that multiple parties—both consumers (clients) and providers (servers) of capa-bilities—must act in concert to achieve an overall goal in a reliable fashion. Each must meet obligations relating to security, performance, and progress. Thus, each requires techniques to overcome various kinds of failure conditions, and to track long-lived interactions that may stretch over days or weeks. Preceding chapters have introduced the basic principles of service-oriented architecture and OGSA (279), explained why future Grid services will have persistent state and active and complex responsibilities (Chapter 17), and introduced the notion of a service-level agreement (SLA) as a means of negotiating expectations concerning the duties that other parties will fulfill (Chapter 18). In this chapter, we introduce client-oriented architectures that may be used to achieve reliable distributed execution in complex, distributed, and dynamic Grid environments. Building on experiences within the Condor Project—which has specialized in the problem of reliably executing jobs on remote machines for almost two decades—we present specific techniques for use in three different contexts: remote execution, work organization, and data output. We first introduce some general design principles that apply to distributed systems and, in particular, to the complex applications often found in Grid computing. We shall encounter applications of each of these principles in our discussion of remote execution, work management, and data output. Effective operation requires responsible behavior. Responsibility is a common …